U.S. Department of Education

The U.S. Department of Education is presented as one of the public institutions drawn into a large-scale attempt to create smaller schools after selective evidence suggested that smallness was associated with educational success. Its participation matters because the episode becomes a case study in how institutions can amplify a compelling but false causal interpretation rather than correct it through comparison and statistical discipline.

Role in the Small-Schools Campaign

The episode began with research into the characteristics of the most successful schools. In a survey of 1,662 schools in Pennsylvania, six of the top fifty were small, an apparent overrepresentation by a factor of four. Those figures encouraged the Gates Foundation to invest about $1.7 billion in efforts to follow up the findings, including the creation of small schools by splitting large schools into smaller units. The U.S. Department of Education joined the campaign through its Smaller Learning Communities Program, alongside other prominent institutions such as the Annenberg Foundation and the Pew Charitable Trust.

The Department therefore appears not as an isolated policymaker but as part of a wider institutional response to a statistical pattern. The reported concentration of small schools among high performers made the proposed reform feel evidence-based, while the participation of major foundations and a federal program gave the interpretation additional authority and momentum.

The Statistical Error Behind the Policy

The central mistake was a failure to ask the corresponding question about unsuccessful schools. A causal story was readily available: small schools could supposedly provide more personal attention and encouragement than large schools, thereby producing higher-achieving students. Yet the evidence did not establish that small schools were better on average. Bad schools, too, tended to be smaller than average. Small schools were more variable, while large schools tended to produce better results, especially in the higher grades, where a wider range of curricular options was valuable.

The U.S. Department of Education’s involvement consequently illustrates the danger of inferring a stable cause from a selected group of unusually successful cases. The apparent success of small schools may have reflected the statistical tendency of small samples to be more variable, making unusually high results more likely as well as unusually low ones. The policy response treated the upper tail of that variation as a reliable model for institutional design.

A Broader Lesson About Organizational Judgment

The small-schools episode belongs to a larger account of judgment under uncertainty. Institutions often construct explanations from the information immediately available, then mistake the coherence of the explanation for the strength of the evidence. A vivid association between small schools and high achievement invited a persuasive narrative, but the omitted comparison with failing small schools changed the meaning of the data.

The episode also contrasts with the procedures recommended elsewhere for improving collective judgment. Independent judgments are most useful when errors are not correlated; open discussion can cause people to follow those who speak early and assertively, whereas collecting brief positions before discussion preserves diversity of knowledge and opinion. Organizations can also use reference-class forecasting, premortems, checklists, and routine review to inspect how a decision was framed, how information was collected, and how conclusions were tested. The Department’s role in the small-schools campaign shows what happens when institutional scale and policy energy reinforce an unexamined interpretation instead.

Significance for Education Policy

The U.S. Department of Education’s participation gives the example consequences beyond an individual’s mistaken judgment. A misleading pattern became the basis for substantial investment, organizational restructuring, and a national education initiative. The resulting lesson is not that educational research is useless, nor that large schools are invariably superior, but that policy must distinguish correlation from causation and examine the full distribution of outcomes rather than only conspicuous successes.

Within the book’s wider argument, the Department’s involvement demonstrates why public institutions need decision procedures that compensate for predictable limits in human reasoning. When a selective statistic supports an intuitively satisfying explanation, institutional endorsement can magnify confidence instead of improving accuracy. Better policy requires asking what the data would show among failures, comparing the proposed intervention with a suitable reference class, and maintaining enough procedural independence to prevent a shared error from becoming a collective commitment.